Cost And Schedule Analytical 
Techniques Development 

Final Report 


November 30, 1994 
NAS8-38784 


Prepared for: 

National Aeronautics and Space Administration 
George C. Marshall Space Flight Center 
Engineering Cost Office 
Marshall Space Flight Center, AL 35812 


Prepared by: 

Science Applications International Corporation 
6725 Odyssey Drive 
Huntsville, AL 35806 





Table of Contents 


I. INTRODUCTION 2 

II. BASIC CONTRACT ACCOMPLISHMENTS 3 

1.0 Task 1: REDSTAR Data Base System Maintenance and Expansion 3 

1 . 1 REDSTAR Data Base System Maintenance 3 

1 .2 REDSTAR Data Base System Expansion 4 

1.3 NASCOM Data Base Development and Expansion 5 

2.0 Task 2: Development of Cost Estimating Techniques 7 

2. 1 Extension and Improvements to NASCOM 8 

2.2 Development of First Pound Cost CERs and Classical CERs 11 

2.3 Training of NASA Personnel 12 

3.0 Task 3: Development of Schedules, Plans, and Requirements 13 

3.1 Schedule Development 13 

3.2 Program Planning 15 

III. ADDITIONAL TASKING 1 6 

1 .0 Ground Operations Cost Model (GOCM) 16 

2.0 PRICE Calibration for NASA Users 16 

3.0 Microgravity Experiments Cost Model (MECM) 17 



I. 


INTRODUCTION 


This Final Report summarizes the activities performed by Science Applications International 
Corporation (SAIC) and Applied Research, Inc. (ARI) (which was purchased by, and integrated into, 
SAIC in January 1994) under contract NAS8-38784 “Cost and Schedule Analytical Techniques Devel- 
opment Contract.” This Final Report is in compliance with Attachment J-2, Paragraph B of the con- 
tract. 


This contract provided technical services and products to the Marshall Space Flight Center’s 
Engineering Cost Office (PP03) and the Program Plans and Requirements Office (PP02) for the period 
August 3, 1991 through November 30, 1994. Detailed Monthly Reports were submitted to MSFC 
which spelled out each month’s specific work performed, deliverables submitted, major meetings con- 
ducted, and other pertinent information. Therefore, this Final Report will not repeat that level of detail, 
but rather, summarize the forty months of contract effort. An executive level summary of accomplish- 
ments is provided in Table 1. 


REDSTAR Data Base 

• Doubled document count from 8,000 to 16,000 

• Developed state-of-art, FoxPro indexing system 

• Installed extensive key word search capability 

• Established seven Special Collections 

NASCOM Hard Copy Data Base 

• Increased contents by 50% to 90 aerospace projects 

• Published 2 updates of data base 

• Added new features: sample problems, data base 
averages, improved documentation 

NASCOM Automated Data Base 

• Conceived, developed, programmed in C++ 

• Delivered operational copies to all NASA Centers 

• Developed and delivered User’s Manual 

• Provided training to NASA personnel 

NASCOM Cost Model 

• Totally restructured and programmed in C++ 

• Delivered initial NASCOM Cost Model 

• Developed NASCOM Cost Model's User’s Guide 

• Contains many innovative features 

Complexity Generators 

• Eliminated subjective engineering judgment 

• Provide traceable, repeatable, documented factors 

Program Planning 

• Wrote five Guideline papers 

• Developed project implementation plans 

• Prepared data requirements packages 


Schedules 

• Added 1,300 schedule documents to REDSTAR 

• Established Schedule Notebooks 

• Developed Schedule Template Evaluation Model 
(STEM) 

• Developed STEM User’s Manual 

• Developed Schedule and Cost Optimization Model 
(SACOM) 

• Developed SACOM User's Manual 

• Developed schedule and logic networks 

NASA Computer Connectivity 

• On NAS A network via T- 1 line 

• Became on-line with MSFC Repository 

• Provided file transfer, and printing at NASA 

Other Analytical Techniques 

• Developed Microgravity Experiment Cost Model 
(MECM) 

• Developed MECM User's Manual 

• Developed Ground Operations Cost Model 
(GOCM) 

• Developed GOCM User’s Manual 

• Developed PRICE System Technical Briefs 

Special Project Support 

• Space Station Redesign Study 

• Access to Space Study 

• New Ways of Doing Business (NWODB) Study 

• Hundreds of Quick Response Actions for MSFC, 
NASA Headquarters, and other centers 


Table 1. Summary of Contract Accomplishments 
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II. BASIC CONTRACT ACCOMPLISHMENTS 


The following sections are broken out by the three MSFC major statement of work task areas 
and provide insight into the work performed in each area over the life of this contract. 

1.0 Task 1: REDSTAR Data Base SystemMaintenance and Expansion 

During the course of this contract, the REDSTAR Data Base was reorganized, doubled in size, 
and upgraded to professional library standards. In addition to improvements to the expansion and 
maintenance of the REDSTAR Data Base, SAIC was instrumental in a major redesign and improve- 
ment of the NASCOM Data Base (NASCOM-DB). The hardcopy version of NASCOM-DB was greatly 
improved with the inclusion of example problems, data base averages, clear project programmatic and 
technical descriptions, and a step-by-step explanation of the NASCOM-Data Base normalization pro- 
cess. SAIC also conceived, designed, and developed the Automated NASCOM-DB (NASCOM-DB). 
This data base allows instant retrieval of any data in the NASCOM-DB and is available to all of NASA. 

1.1 REDSTAR Data Base System Maintenance 

SAIC provided approximately 1,000 square feet of floor space, 91 file cabinets, and 27 book- 
cases for the REDSTAR Data Base. The REDSTAR Data Base was under the management of a mas- 
ters’ degreed librarian who is listed in the current publication of Who’s Who Among National Technical 
Libraries. Not only has the sheer volume of REDSTAR source documentation increased twofold in the 
last three years (average yearly document entry rate of nearly 2,700), but library organizational stan- 
dards, document retrievability and accountability, and interfaces to nationwide document repositories 
are now on par with any professional library of this nature in the country. 

The most valuable of historical documentation is worthless if it can not be retrieved from hard 
copy of automated filing systems. To this end, SAIC has instituted and maintained a high degree of 
professional library standards in its management of REDSTAR. When the database was transitioned to 
SAIC, the Library Reference System (LRS) was non-standardized and in need of updating. The LRS, 
then coded in dBase III, could not be queried on document keywords, search routines required an in- 
depth knowledge of dBase code, and data characterization was incomplete in that only a limited num- 
ber of NASA programs were on the existing data characterization form. 

SAIC modernized the LRS and thus enhanced the utility of REDSTAR. The data characteriza- 
tion sheet or “cut sheet”, which is the hard copy representation of the LRS’s electronic query matrix, 
was greatly expanded. (See Figure 1-1 for portions of the updated “cut-sheet”.) It now includes about 
50 additional missions and attached payloads. A host of topics in the field of management and engi- 
neering efficiencies that are of current and future interest to NASA such as, NWODB, Skunkworks, 
total quality management, and concurrent engineering have also been added. Further, data can now 
characterized as pertaining to ground, launch or mission operations. These recent updates to the “cut 
sheet” provided a more comprehensive way to enter and retrieve data from REDSTAR. SAIC also 
established a keyword REDSTAR LRS search capability. This powerful tool is easily maintained and 
allows data retrieval by simply keying in a one-word topic. SAIC and NASA personnel were provided 
with a list of keywords that were standardized for use in the LRS routines. 

The updated LRS was programmed in the FoxPro data base application. Because FoxPro oper- 
ates in the Windows environment, REDSTAR’s LRS search and retrieval commands are now “point 
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and click” with search times 
nearly instantaneous. Typing 
in dBase code and waiting up 
to 3 minutes for search re- 
sults are a thing of the past. 
This dynamic increase in 
search speed is even more 
important when one consid- 
ers the rate at which the size 
of the REDSTAR Data Base 
is increasing. Another library 
service that SAIC has intro- 
duced to this contract is the 
use of both local and national 
information centers as 
sources of aerospace infor- 
mation. For example, we 
have established working re- 
lationships with the historians 
Figure 1-1 "Cut Sheet" Provides Comprehensive and archivist at MSFC, JPL, 
Data Point Characterization Goddard Space Flight Center 

(GSFC), the University of 

Alabama in Huntsville, and the Air University Air Command and Staff College at Maxwell Air Force 
Base. Through these sources, as well as the Redstone Scientific Information Center (RSIC) and MSFC 
Repository, we have obtained much valuable historical data in support of the CSATD contract. We are 
on-line with the MSFC Repository’s automated data retrieval system and developed search strategies 
of this large information center from the 486 PC in REDSTAR. 

Also coded in FoxPro was the REDSTAR document checkout program. Checkout procedures 
were standardized and implemented. The REDSTAR document number, document title, the borrower’s 
name, and the date of checkout were input into the electronic file. When documents were returned, the 
date of return was input into the checkout routine. 

The REDSTAR inventory was used to ensure document accountability, to restructure the con- 
tents of each filing cabinet to make room for future documents, and to assess the feasibility of creating 
new special collections for data of the same subject. On completion of each end of the year REDSTAR 
inventory, a complete to-date listing of the contents of REDSTAR was published and sent to resource 
and planning groups throughout NASA. 

1.2 REDSTAR Data Base System Expansion 

In the past three years, the REDSTAR document count has increased from about 8,000 to nearly 
16,000. This statistic is astounding when one considers that SAIC has managed REDSTAR only 3 of 
its 23-year existence. 

For each new data collection effort, in-house sources were reviewed and contacts made to 
likely NASA and industry sources for information and leads to relevant data. This approach has worked 


REDSTAR DOCUMENT DATA INPUT FORM 
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well for S AIC over the past three years. For example, our collection of Space Station Freedom, CRRES 
and Orbital Maneuvering Vehicle (OMV) project data was obtained through our established relation- 
ships with personnel in these NASA project offices. Approximately 500 documents pertaining to these 
3 projects were obtained before the project offices were closed or relocated. Figure 1-2 contains a 
listing of some of the data collection contacts made for MSFC in the past three years. 

Still other data was obtained by consulting the LRS, RSIC, the MSFC library and repository, 
DIALOG Information Services, NASA Scientific and Technical Aerospace Reports, the Defense Tech- 
nical Information Center, the National Technical Information Center, the National Space Science Data 
Center and other available data bases. 

In the past three years we made numerous data collection trips including those to GSFC, JPL, 
KSC, JSC, NASA Headquarters, Reston, Crystal City and El Segundo, California. These trips were 
fruitful in that we obtained over 25 mission costs and technical data packages in sufficient detail to 
include them in the NASCOM-Data Base and associated cost models. 


REDSTAR Special Collections, established by SAIC, allowed the analyst to conveniently pe- 
ruse data of the same subject matter. This allows a REDSTAR user to go to one or two file cabinets that 
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Company, Contact, Data Requested 



Aero Astro, Inc., Melanie Horn, ALEXIS & HETE 
Aero Astro, Inc., Richard Warner, ALEXIS & HETE 
Aero Astro, Inc., Richard Warner, ALEXIS & HETE, 2 
BMDO Edwards Airforce Base, Jason Feige, MSTI 
Boeing Marketing, Seattle, Barbara Murphy, Boeing 777 
Boeing, Diane Copenhaver, IUS launch cost 
Boeing, Roy Berg, Minuteman II cost 
Fairchild Aircraft, Don Johnson, Geowarn question 
Fairchild Space, Bob Bartlett, EUVE 
Fairchild Space, Tom Moser, EUVE 

General Dynamics, John Silverstein, Titan IV Centaur launch costs 

General Electric, Dave Lane, Engine cost data 

GSFC Historian, Keith Koehler, document info 

GSFC, Bill Tragesar, EUVE & XTE 

GSFC. Bob Meiis, TOMS 

GSFC, George Barth, TRDS 

GSFC, Jim Sarrowman, EUVE & XTE 

GSFC, Jim Manion, EOS 

GSFC, Mike Weiland, GOES 

GSFC, Mildred Saari, UARS 

GSFC. Paul Caruso, ISTP 

GSFC, Steve Dobroseilski, METSAT 

GSFC, Vicki Corey, get on mailing list of GSFC document 

Honeywell Library, Unda Tabor, EOS computer costs 

Honeywell, Inc., Don Lee, FEWS 

Hughes Aircraft, Uldis Lapins, HS-376 info 

Hughes Space & Communications Group, Ed Limburgh, UFO 

Hughes Space & Communications Group, Jerry Adams, UFO 

IBM, Mark Michrana, Shuttle & SSF On-Board Processing 

Innotech Aviation, Larry Neily, Geowarn question 

Jet Propulsion Laboratory, Robert Metzger, MSTI 

JPL Archives, Julie Reiz, JPL spacecraft document lists 

JSC History Office, Joey Kuhlman, POP s documenl info 

JSC Lambda Point Office, Joe Malloy, get on mailing list for document 

Lockheed Research & Development Division, Don Efene, CRSS 

Lockheed Research & Development Division, Dr. Ralph Kuiper, CRSS 

MacDonald Douglas, Fred Tolley, F-15E landing gear cost 

MacDonald Douglas, Jim Meyers, PAM, Delta Stage II launch cost 

Martin Marietta Astro Space, Don Brown, Milstar 

Martin Marietta Astro Space, Jerry Miranda, Satellite Processing 

Martin Marietta Astro Space, Marvin Kravitz, DSCS-3 

Martin Marietta Astro Space, Walt Woessner, DSCS-3 

Martin Marrieta, Larry Price, Transtage launch cost 

Morton Thiokol, Don Wilson, Castor nozzle cost 

Morton Thiokol, Jack McCommons, STAR 30E, 48B cost 

Morton Thiokol, Jack McCommons, Star 48. 37 motor cost 

Morton Thiokol, Dennis Walstrum, Star 1 7 motor cost 

MSFC, Gerry Hall, EASE costs 

MSFC, Keith Hetner, AXAF 

NASA Chief Historian, Dr. Launius 

OSC. Chris Schade, Pegasus & Taurus cost 

OSC, Jeff Campbell, TOS launch cost 

Phillips Laboratory, Jim McClellan, MSTI 

Pratt & Whitney, Mark Sullivan. Engine cost data 

Pratt & Whitney, Nancy Colaguori, RL10-A-4 

Rolls-Royce, Robert Stangarone, Engine cost data 

Spectrum Astro, Ray Rew, MSTI 

Tecolote, Mike Mahoney. Conestago cost 

Teledyne Systems Co., Roger Gilreath, Milstar 

TRW Space & Electronics Group, Susan Brough, DSP 

U. S. Navy, Dick Coffman, UFO 

U. S. Navy, Steve Archer, UFO 

US Navy Research Lab, Paul Regeon, Clementine 



Figure 1-2 SAIC Has Many Data Collection 
Contacts Across the Country 


contain the majority of the REDSTAR data on a 
specific subject. Special collections were set up for 
NASCOM-Data Base data points, schedules, Pro- 
gram Operating Plans (POPs), Space Station, 
launch vehicles, solid rocket motor. Shuttle 
projects, unmanned spacecraft, manned spacecraft, 
and the External Tank. 

SAIC placed great emphasis on quick and 
accurate responses to sudden data needs and re- 
quests by MSFC. We efficiently provided rapid 
response data and analyses for studies such as: New 
Ways of Doing Business (NWODB), Space Sta- 
tion Freedom Redesign, Access to Space (includ- 
ing single and two-stage-to-orbit configurations), 
Spartan conversion project, small launch vehicle 
data collection, and National Launch System 
(NLS). These quick turnaround responses consisted 
of data set formulation, CER development, cost 
comparisons and trends, spreadsheet development, 
cost driver identification, and generation of hard- 
ware estimates. 

1.3 NASCOM Data Base Development and 
Expansion 

The NASCOM-DB consisted of 62 space- 
craft project cost and technical analyses when SAIC 
was awarded the CSATD contract. SAIC has ana- 
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lyzed 42 projects for the NASCOM-DB, including new analysis for 28 projects and the re-analysis of 
14 of the original data points to provide component level visibility. In three years SAIC was respon- 
sible for 42 of the 90 total NASCOM-DB projects or 47%. 

The NASCOM-DB is essentially the very best and most complete aerospace project cost, tech- 
nical and programmatic data from the REDSTAR Data Base. SAIC published Version 2.0, in January 
of 1993, and Version 2.1, in December 1993 and distributed them to all NASA centers. 

Enhancement of the NASCOM-DB was a dynamic process at SAIC. We ensured that the con- 
sensus opinion PP03 as to the proper ground rules and process have adhered to in our effort to continu- 
ally improve the NASCOM-DB. Individual study efforts including verification, modification, alterna- 
tives presentation, and joint working sessions with PP03 have consumed approximately 2,000 man- 
hours of analytical effort. This entire effort was extremely successful, producing a consistent, standard- 
ized NASCOM-DB methodology. 

The NASCOM-DB “data was normalized to a consistent set of ground rules and assumptions 
that govern the presentation of three separate and distinct sets of cost analyses (“As Reported”, “Mod- 
eled” and “Synthetic Wraps Applied”). All historical cost data within the NASCOM-DB were analyzed 
for inclusions and exclusions and then converted to a standard constant year dollar from real year dollar 
project cost reports. All technical data was normalized to consistent terms and units. 

The WBS for all NASCOM-DB projects included the actual project WBS at lower levels, yet 
applied generic accumulators at the subsystem and system level. This WBS normalization approach 
affords the MSFC cost engineer the option of using specific analogies or more generic elements when 
developing subsystem data sets for cost estimates. Component, subsystem, and system level mass 
properties which are all converted to pounds, conform to the same WBS. 

The design approach for all NASCOM-DB data points is either prototype or protoflight. This 
distinction has predictable impact on project cost and is reported as such in the “As Reported” data. 
Consistent handling and normalization for the cost associated with the dedicated test article (prototype 
approach) versus all cost associated with test and refurbishment of the protoflight article was strictly 
maintained in the S AlC-produced NASCOM-DB. 

Stratification of the NASCOM-DB was by mission or project type, i.e., manned spacecraft, 
unmanned spacecraft, and launch vehicle stages and liquid rocket engines. Also separately included 
were scientific instruments grouped by classes based on the specific function of each. 

The final step in the NASCOM-DB methodology was the calculation of first pound costs. In 
NASCOM-DB terminology, first pound costs are the cost for the first pound of hardware. Statistically, 
first pound costs are the y-intercept values for a power curve equation, defined by the formula: y = 
ax A b, where y equals costs, x equals weight, b equals the slope of the curve, and a equals the y-intercept 
or the first pound cost. This formula requires that the b-value be known before the equation can be 
algebraically transformed and exercised. B-value determination is relative to the type of aerospace 
hardware, and mechanical/electrical composition. SAIC introduced subsystem unique b-values in the 
publication of NASCOM-DB Version 2.0 after exhaustive study and research. These unique b-values 
now reflect a logical and verifiable impact on a-value calculation, attributable to the degree to which 
economies of scale are exhibited. Technical, contractual, and programmatic resumes containing cost 
driving parameters, major milestone, known inheritance and hardware compatibility, and major pro- 
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grammatic influences were established (if required) or reverified and maintained for each data point in 
the NASCOM-DB. These resumes were presented in project and subsystem form, and follow the 
NASCOM-DB cost and weight WBS. 

SAIC created an innovative new tool with which all NASCOM-DB information can be electronically 
retrieved. The Automated NASCOM-DB (NASCOM-ADB) is a user friendly data retrieval tool that 
operates in the Microsoft Windows environment and is coded in C++. The NASCOM-ADB allows 
users to electronically search, filter, and retrieve cost and technical data at any level presented in the 
hardcopy NASCOM-DB. The data level selection screen from the NASCOM-ADB is shown in Figure 
1-7. The data contained in the NASCOM-ADB is consistent with the data contained in the NASCOM- 
DB, Version 2.1. NASCOM-ADB was distributed to all NASA Centers in December 1993. 

The NASCOM-ADB was segregated into eight data base files including: Structures; Electrical Power; 
Command, Command, Communication and Data Handling; Attitude Control; Environmental Control 
and Life Support; systems Level; and, Miscellaneous. These data files include over 135 different fields 
of information for searching and filtering including 1 1 cost, 3 weight, and 121 performance, technical 
and programmatic fields. 

Numerous special features contained in the NASCOM-ADB combine to make it a very convenient and 
useful tool for establishing cost estimating data sets. Search results can be saved and reopened to 
append or delete data points. Data can be easily copied and pasted into other Windows applications. 
Data can be printed to any printer. Font styles and sizes of the data can be changed. Columns and rows 
of data can be hidden from view or from being printed. The mission field and the WBS item field can 
be “frozen” on the screen to be viewed at all times while scrolling through the data outside the display 
boundary. A “Filter Reporter” shows what filters were used to complete the search. A Help Index is 
provided to supply information about any data point (program) of interest. 

2.0 Task 2: Development of Cost Estimating Techniques 

While this task is of equal importance to the REDSTAR Data Base maintenance and expansion 
task, the work performed on the cost estimating task has a more profound and visible effect on the cost 
analysis results produced by the Engineering Cost Office. S AIC’s experience in developing innovative 
and defensible cost estimating techniques for NASA was directly applied to completing the initial 
version of the NASCOM Cost Model. 

SAIC’s cost estimating techniques development efforts for other NASA customers increased 
our NASA experience and understanding and provided synergistic efforts that benefited the Engineer- 
ing Cost Office and the Program Plans and Requirements Office. For example, the SAIC development 
of the Microgravity Experiments Cost Model (MECM) led to three cost effective innovations for PP02 
and PP03. The synergistic MECM activities are: (1) The shell of the MECM model was used to de- 
velop the NASCOM-Lite program in slightly over a month. To develop this powerful tool from scratch 
would have taken many months. (2) The development of the schedule estimating portion of MECM 
using microgravity schedule templates was the precursor for the Schedule Template Evaluation Model 
(STEM). (3) The cost and schedule integration techniques used in the MECM were used in the devel- 
opment of the Schedule and Cost Optimization Model. 
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2.1 Extensions and Improvements To NASCOM 

The NASCOM Cost Model is an automated cost model that provides computer access to the 
NASCOM-DB to allow cost engineers to select most analogous data points for estimating DDT&E, 
flight unit cost, and integration cost elements. The NASCOM Cost Model relates the technical and 
programmatic aspects of a space or ground based element to its estimated cost. The model is capable of 
estimating cost at any level of definition (Phase A through Phase C/D), and at any level of detail (total 
project to component level). 

As a place-holder until the NASCOM Cost Model was on line, SAIC developed and delivered 
a precursor model called NASCOM-Lite. This model was completed in May 1994 and training pro- 
vided to MSFC users at that time. In August 1994, an updated User’s Manual was delivered to NASA 
and a training class was provided by SAIC to NASA Headquarters Comptroller Office personnel in 
Washington. NASCOM Lite is versatile, automated, user friendly, and data backed. It was Microsoft 
Excel based for use on Macintosh or PC computers. The model allows component or subsystem level 
cost estimating based on the NASCOM Data Base and using either data base averages or a specific data 
point (analogy estimating). This model, unlike the NASCOM Cost Model, requires user-defined com- 
plexity factors as inputs. It contains a thru-put option for D&D, system test hardware, and/or flight unit 
cost which may be exercised if appropriate. NASCOM Lite automatically calculates system level or 
“wraps” costs. It includes a schedule estimating capability and shows results in real or user-defined 
fixed year funding spreads. 

At completion of the current contract the automated NASCOM Cost Model Windows-based 
shell was completed and delivered to MSFC. Many of the NASCOM Complexity Generators were also 
completed and delivered to MSFC. The automated NASCOM Cost Model provides the capabilities 
shown in Table 2-1. 

SAIC’s innovative concept of NASCOM Complexity Generators brings MSFC’s cost model- 
ing capabilities on par with any commercially available cost model (i.e. PRICE and SEER). The major 
advantages of this approach over the commercial models are (1) the data base used to develop the 
NASCOM Complexity Generators is known and consists strictly of NASA and Air Force aerospace 

hardware (unlike commercial models whose data 
bases and methodologies consists of little space 
hardware and many non-government programs); (2) 
the difficult, uncertain, time consuming, and costly 
process of calibrating the commercial models is 
built into the NASCOM Complexity Generator 
design; and (3) the knowledge that the MSFC cost 
engineers have about the programs in the 
NASCOM-DB can be directly applied to the use 
of Complexity Generators. 

The Complexity Generator was smoothly 
integrated into the NASCOM Cost Model estimat- 
ing process by simply being an alternative to the 
use of engineering judgment in developing com- 


• On-line access to all NASCOM-DB data 

• Integrated and operational Complexity Gen- 
erators (subsystem and component levels) 

• User defined production quantity and quantity 
next higher assembly 

• Automatic or user defined application of 
integration costs 

• The ability to throughput costs, enter user 
defined CERs, or percentage cost down to a 
lower WBS level 

• A user defined WBS with an unlimited num- 
ber of levels 

• Automated generation of output reports 


Table 2-1 NASCOM Capabilities 
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plexity factors. The MSFC engineer may choose to use a Complexity Generator or a user defined 
complexity for each CER used in an estimate at the subsystem or component level. If the user chooses 
to use a Complexity Generator, a screen similar to the one shown in Figure 2-1 will be shown. The top 
half of the screen will display the performance parameters that are included in the generator. The 
middle section of the screen displays the programmatic parameters that have been normalized from the 
data base. The bottom portion of the complexity Generator screen shows the Design and Development 
(D&D) and flight unit complexity values derived from the performance and programmatic inputs. 


When an analogous data 
point is chosen from the 
NASCOM-DB, the values 
shown in the performance and 
programmatic portions of the 
screen reflect the complexity of 
that data point (or the average 
of several data points if mul- 
tiple data points are chosen). If 
none of the complexity vari- 
ables are changed the complex- 
ity factors for D&D and flight 
unit will equal 1.0. This will 
give the same result as using the 
NASCOM-DB to calculate an 
assumed slope “a-value” and 
using a user defined complex- 
ity of 1.0. If one or more of the complexity variables are changed to more closely represent the com- 
plexity of the hardware to be estimated, the complexity factors will change accordingly. 

The NASCOM Complexity Generators were based to a degree on a methodology developed for 
JPL called Cost Complexity Relationships (CCRs). The CCRs are more similar to the classical CER 
development approach where least squares regression methods are used to develop a cost estimating 
equation. They differ from normal CERs in that the complexity of each data point was normalized into 
several categories and removed from the cost. The remaining cost is said to be the “null complexity 
cost” and was used in a weight based regression. To estimate a new project, the weight based equation 
was used and the complexity of the new project was added back using the same factors used in the 
normalization. This approach will not work with the NASCOM Cost Model without some modifica- 
tion due to the “first pound” cost estimating methodology and the selection of most analogous data 
points. SAIC’s five step process used to develop the existing Complexity Generators is described be- 
low. 


Power Generation Complexity Generator 


Performance Measures 


System Type 


Battery Type 


K3H 


Weight 


Power Output 


Amp-Hours 







Power Output 


Programmatic Measures 


Specification 


K> 


Heritage 


Req. Changes 




Interfaces 


K> 


Experience 


K> 


New Design 


KH 


DDT&E 

Complexity 


0.785 


Unit 

Complexity 


0.893 


[ Return ] [ Cancel ) 


Figure 2-1 Complexity Generators Subsystem Example 


STEP 1: Define Categories. The parameters that are appropriate for the Complexity Genera- 
tors were identified. These parameters vary between subsystems. Most of the programmatic param- 
eters were consistent between subsystem classes, but the technical parameters always vary. 

The technical categories are parameters that can be measured or identified based on the design. 
Examples of technical parameters used in the NASCOM Complexity Generators include: weight, out- 
put power, solar array surface area, number of transmitters, bandwidth, level of redundancy, battery 
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life, specification level, construction method, and storage capacity. The programmatic category in- 
cludes parameters that are not as easily measured. Examples of the programmatic parameters include: 
inheritance, technology, new design, testing level, technology readiness level, design life, require- 
ments changes, experience, schedule impacts, and interfaces. 

STEP 2: Establish Category Relationships. This step in the Complexity Generator develop- 
ment process involved the application of regression analysis techniques to define the “curve” that best 
fits each of the complexity parameters used in a particular generator. Our experience developing the 
JPL CCRs proved invaluable in this process. The range of possible values for each of the complexity 
parameters was assigned a weighting factor based on its affect on the total cost of the element. Rela- 
tionships were determined using regression methods, experience, and engineering judgment. The types 
of equations used include: linear functions, logarithmic functions, exponential functions, step func- 
tions, and combinations of functions. An example of the application of these multiple functions can be 
seen in Figure 2-2. 

STEP 3: Determine “Null” Cost Equation. The determination of the “null” cost equation 
used the same approach developed for JPL. The complexity was normalized out of the data base using 
the category relationships established in STEP 2. This step often required specific data collection and 

analysis to determine why the nor- 
malization of select programs are 
not behaving as predicted. We 
found that “outliners” in this ap- 
proach can be explained and cor- 
rected by additional data collec- 
tion and analysis. 


STEP 4: Test, Validate, 
and Document. Before any of 
the cost complexity relationships 
were used in the NASCOM Cost 
Model, they were tested by SAIC 
using actual data not included in 
the development process. After 
thoroughly testing the generators 
the documentation was devel- 
oped. The documentation include 
the data base used and the statis- 
tical validity of each of the com- 
plexity parameters and the “null” 
cost equation. It includes validity 
ranges of all parameters and rec- 
ommendations of best results of 
all parameters and recommenda- 
tions of best areas in which to use 
the generator. 

Figure 2-2 Complexity Element Relationship Example 
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STEPS: Incorporation Into The NASCOM Model. The final step in the development of a 
NASCOM Complexity Generator was including it in the NASCOM Cost Model. This process in- 
volved adding the complexity equations to the NASCOM Cost Model and adding the normalization 
data for each data point into NASCOM-ADB. Both of these are required for the Complexity Generator 
process to work in the NASCOM Cost Model “first pound” cost environment. 

2.2 Development of First Pound Cost CERs and Classical CERs 

Very often homogenous cost data for similar NASA hardware does not exist because of varied 
cost reporting formats, differences in contract types, budget and scheduling constraints, redesigns or 
major engineering changes, and refocused mission requirements. The first pound cost estimating ap- 
proach was developed to allow CER development with one or few data points due to this non-homoge- 
neous data. 

In the past three years SAIC has developed many CERs for NASA and improved and expanded 
the NASCOM-DB which is used to create CERs for NASA. We used the first pound cost methodology 
for the majority of the CERs produced for the Engineering Cost Office using the NASCOM-DB. We 
used classical CER approaches for many special analyses for MSFC. We also used classical regression 
methods for the development of the complexity functions in the Complexity Generators and for the 
development of all the CERs in the MECM model. 

The development of a first pound costs CER required meticulous data analyses and normaliza- 
tion. These data were normalized to a set of ground rules consistent across the entire NASCOM-DB. 
Six major stratifications of the NASCOM-DB exist. They are: manned spacecraft, unmanned earth 
orbital spacecraft, unmanned planetary spacecraft, launch vehicle stages, liquid rocket engines, and 
scientific instruments. The economy of scale slope of b- value is selected from a table of values specific 
to subsystem type. This table was derived by SAIC after analyzing over 1,000 Classical CERs devel- 
oped for all NASA centers over the past 15 years. 

SAIC has structured NASCOM-DB first pound cost CERs such that they may be used in a 
variety of ways. Analog estimates may be created by using the weight of the hardware to be estimated 
and the most analogous NASCOM-DB data point a-value in the first pound cost equation (cost = a * A 
b). We also created tables of data points within each of the six NASCOM-DB data classes. 

The first pound cost CER methodology is consistent with the SAIC-developed NASCOM Com- 
plexity Generator. The results of any NASCOM-DB first pound cost calculation, or least squares mul- 
tiplicative, additive error, or “dummy variable” CER can be adjusted by any of the components of the 
Complexity Generator. An example is the Technology Readiness Level (TRL) curve recently devel- 
oped for MSFC and shown in Figure 2-3. The trend shown in this curve allows adjustments for CER- 
estimated cost relative to the availability of the technology to be used in the hardware to be estimated. 

In addition to the first pound costs estimating techniques, SAIC developed classical CERs in 
support of the NLS study, Access to Space, MECM cost model. Space Station Redesign, and rapid 
response items. PRC-Reg, GSFC-Reg, and Excel are some of the regression packages used by SAIC 
analysts. In fact we modified the program code for both PRC-Reg and GSFC-Reg to improve the 
output format and printing capabilities of these regression tools. In these and other related efforts, 
SAIC analysts employed standard methods to evaluate and verify all CER results such as the standard 
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error, the “t” statistic, 
the “F” test, correla- 
tion coefficient, etc. 

The final 
steps in the SAIC ap- 
proach to classical 
CER development 
was to determine re- 
sults, draw conclu- 
sions and complete 
documentation. Con- 
tained in the docu- 
mentation of CER 
development are the 

data points used, complete definitions of the cost content of the CER, the data analysis results, graphi- 
cal display of the CERs with labeled data points and curve fit lines, and conclusions with CER recom- 
mendations and CER applicability ranges. 

2.3 Training of NASA Personnel 

The complex nature of the cost estimating models and techniques developed under this contract 
call for comprehensive documentation and training. The importance of the results derived from these 
models make it essential that users are knowledgeable of the model capabilities, limitations, and as- 
sumptions made during development. 

SAIC has provided this type of documentation on numerous occasions in the performance of 
this contract. Documentation examples are listed in Table 2-2. 



Figure 2-3 Technology Readiness Level (TRL) Example 


• Updated NASCOM-DB documentation 

• User's Guide for NASCOM-ADB 

• User's Guide and a two volume set of model documentation for MECM 

• User's Guide and documentation for GOCM 

• User's Guide for the STEM 


Table 2-2 SAIC Developed Documentation 

Having a well written, complete set of documentation for a cost model is often not 
adequate information to get a fast start in using the tool. For this reason we prepared and presented 
training material for several of the models developed under the CSATD contract. Table 2-3 lists some 
examples of NASA training performed by SAIC. 

The SAIC provided training included hands-on computer use, training presentations, example 
problems illustrating cost and schedule estimating techniques, and handout material with quiz prob- 
lems. 
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• Two day training session on the NASCOM-DB and NASCOM-ADB 

• Two classes of one half day training in the use of the MECM cost model 

• Training session on the development approach and uses of the NASCOM-Lite model 

• NASA Headquarters training on the NASCOM-DB, NASCOM-ADB, and 
NASCOM-Lite 


Table 2-3 SAIC NASA Training 

3.0 Task 3: Development of Schedules, Plans and Requirements 

SAIC greatly expanded the Schedules, Plans, and Requirements task under the CSATD contract. We 
placed greater emphasis on the schedule analysis by initiating an aggressive program resulting in the 
accumulation of almost 1,300 documents containing historical aerospace schedules, establishment of a 
major Schedule Collection in REDSTAR, creation of a Schedule Notebook, and development of auto- 
mated schedule models. 

SAIC also has expanded the program planning support role. Work included the development of 
implementation and project plans. Guidelines development, and WBS and logic network preparation. 
We provided independent reviews and assessments of NAS A program planning, procurement and tech- 
nical management documents. 

3.1 Schedule Development 

Our scheduling skills were demonstrated in the CSATD contract by the development of sched- 
ules for the Space Station Hab Module, SEI, Spartan Missile, Nuclear Propulsion Testing and four 
different schedules for the Access to Space Study. Further, we analyzed the NASA Strategic Plan and 
pointed out schedule inconsistencies as well as developed a NASA “new start” schedule and a long 
range launch schedule for the plan. SAIC also developed a presentation on schedule slip factors for 
PP02 and prepared numerous letters which were sent by MSFC to project offices and major aerospace 
contractors requesting historical schedule data. 

New documents containing historical schedule data were obtained from NASA individual con- 
tacts, other government agencies, and hardware development contractors. The new schedules, along 
with existing ones, were made into separate Schedule Collection section of REDSTAR. Additionally, 
an extensive Schedule Notebook of over 1,000 schedules on 77 aerospace programs was developed 
that contained all the basic schedules collected. The schedules were categorized according to twelve 
classifications (unmanned vehicle systems, solid motors, etc.) provided by PP02. Using this data base 
SAIC developed historical schedule templates for project categories that indicate typical or average 
schedule duration between milestones. 

Under the contract, SAIC also developed charts on changes in NASA processes and their im- 
pact on NWODB. These types of data were researched by SAIC for their impact on NASA program 
planning. We also reviewed the literature to identify projects where major successes have been ob- 
tained in the development of reduced schedule development time. 

SAIC developed the Schedule Template Evaluation Model (STEM) which is a dynamic 
model that was based on historical schedule data. It is operational and two versions of a User’s Guide 
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have been published by SAIC. STEM allows the creation of new project generic schedules using the 
historical data base inherent in the model, or it allows the user to manipulate milestones to develop his/ 
her own schedules. An example template screen from STEM is shown in Figure 3-1. 

Aerospace pro- 
gram costs and sched- 
ules are tightly coupled 
to each other, and both 
are strongly dependent 
on technical require- 
ments. However, the 
parametric linking of 
cost and schedules has 
always been a rather dif- 
ficult task. This is due in 
part to the fact that cost 
data tends to break down 
into various levels of 
hardware end items and 
system type functions, 
while schedules are sub- 
divided into program 
milestones and other cal- 
endar related events. These two different breakouts are not easily crosswalked to allow cost to be 
shown on schedules, or vice versa, with any high fidelity of detail. 

SAIC made much progress in this difficult linking area in the currently operational, automated 
MECM model. Costs werre time-phased based on actual microgravity historical schedule milestone 
data which we collected, normalized, and developed into individual TERs for microgravity programs 
of varying complexity. While the average development and unit production schedules are known, judg- 
ment must be applied as to the distribution of cost within the given time period. This was typically done 
at the subsystem level using beta spreading functions. 

Non-optimized schedules definitely impose cost penalties that should also be assessed. SAIC 
accounted for this phenomena in MECM through establishment of a curve relating cost savings and 
penalties to schedule duration in terms of percent from an established CER-output cost. This curve was 
based on relationships from commercially available parametric models. We found that microgravity 
cost CER outputs were tied to a schedule which normally exceeded the optimum schedule by 20 per- 
cent. 
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Figure 3-1 STEM Provides Schedule Templates for a 
Variety of Space Missions 


Cost and schedule relationships have also been included in the automated NASCOM-Lite Cost 
Model which was developed under this contract. In that model, the STEM templates are called up on 
the screen and the appropriate one selected for the type of program. Time-phasing is then based on that 
template using user-determined beta distributions at the subsystem level. This model does not have the 
schedule-driven cost penalty feature of MECM. 

In the last months of the CSATD contract, an additional automated cost-optimizing schedule 
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tool was programmed called SACOM (Schedule And Cost Optimization Model). This model uses, as 
input, the costs developed in NASCOM-Lite (or NASCOM Cost Model when operational at contract 
end). It allows the individual major program elements, with their associated costs, to be moved about 
on the overall schedule until a program funding profile is obtained which best matches the expected 
program budget and minimizes cost and manpower peaks and valleys. Limited program logic was built 
in to ensure the integrity of the program is maintained while these schedules are moved. For example, 
the software will not allow system tests to occur before the system test hardware is built, or schedule 
compression beyond reasonable expectations. This model operates at the subsystem level and provides 
graphic display of funding, overlaid on the schedule, as well as, numerical output. Preliminary cost 
penalties associated with non-optimum schedules were incorporated in the SACOM model. 

SAIC developed time-phased generic logic networks utilizing several different techniques. We 
used the powerful PC-based ARTEMIS 7000 software and hardware including a 36 inch roll-fed 8 pen 
plotter to support PP02 requirements under the contract. We have also used “Finest Hour” in the prepa- 
ration and development of generic logic networks for the generic unmanned launch vehicles and the 
Laser Beamed Power Study during the contract. We also developed a logic network for the “NASA 
Approval Budget and Procurement Cycles”. In addition, we developed schedule logic that tied the 
NASA project planning activities with OMB circular A- 109 and the budget cycle. Microsoft Project 
has been used to develop logic networks for other MSFC projects as has Open Plan. Each of the 
software systems has its own set of advantages and disadvantages. SAIC has developed “work arounds” 
for many of these including color coding and hand manipulation of the network to avoid over laid lines 
or lines underrunning unrelated activity boxes. Time-phasing of major logic networks was accom- 
plished by indicating time durations on the activity boxes. 

3.2 Program Planning 

SAIC prepared program planning documentation and reports relating to project management 
functions, management topics, and other program planning related subjects as requested by PP02. For 
example, SAIC prepared draft implementation plans for the LUTE Project. This plan was then used by 
the LUTE Project team to finalize their planned approach that was submitted to NASA Headquarters 
for approval. Other examples of program planning documentation developed during the contract in- 
clude several seminars (currently called guidelines) prepared for use by PP02. Some topics that we 
were researched, developed, and presented are Project Plans, WBS, Management Directives, Project 
Planning, and NASA Agreements. 

Another important program planning task that was accomplished during the contract was the develop- 
ment of Phase A and B data requirements packages. SAIC developed packages for both in-house and 
contracted Phase A and Phase B studies. The packages were developed to be consistent with Phase C / 
D requirements which were the responsibility of MSFC’s Science and Engineering Directorate. 

SAIC has a great deal of overall experience and involvement in the program management functions. 
Our unique NASA experience covers the total gamut from early NASA programs of the 1960s to the 
latest NASA programs and projects. We understand which functions and topics are important. We have 
prepared many handbooks, guidelines, and presentations in the past and will continue to do so. We 
know the basic NASA Directives System and how changes are made to it. We also understand what is 
needed and how to get prior practices and other documentation that we will use to prepare new guide- 
lines and presentations. Our personnel have established working relationships with many of the current 
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and former NASA program and project managers, and are experienced in interviewing and obtaining 
information from them. We also have technical writing skills, graphic capabilities, and presentation 
talents that will be brought to bear in the development of presentations. 

III. ADDITIONAL TASKING 

In addition to the mainline tasks accomplished for the Program Development directorate of 
MSFC, three in-scope tasks were performed under the contract for other NASA elements. These were 
the development of the Microgravity Experiments Cost Model for the MSFC Microgravity Projects 
Office; the development of the Ground Operations Cost Model for the Kennedy Space Center (KSC); 
and the calibration of the PRICE Systems Cost Model for NASA users, directed by the NASA Head- 
quarters Comptroller Office. Some of the synergistic elements of these activities have already been 
mentioned in the discussion of contract accomplishments for MSFC, but specific work performed is 
described in the following paragraphs. 

1.0 Ground Operations Cost Model (GOCM) 

SAIC completely reworked the old GOCM that had been developed for KSC by another con- 
tractor. The original model would only estimate Shuttle ground operations and lacked documentation, 
flexibility, operating speed, simplicity, and overall credibility. The model was reprogrammed by SAIC 
as a menu-driven spreadsheet model that estimates facilities, schedules, manpower and costs at many 
levels and provides reporting, charting, supporting data bases, and documentation. It will estimate up 
to two different type vehicles in flow at KSC, which is another major enhancement. The model will 
estimate ground operations cost of over 80 different combinations of current and future launch vehicle 
elements including expendable, partially reusable, and reusable concepts. 

SAIC totally rebuilt the data base using newer Shuttle actuals than were in the old model. A 
more uniform and inclusive WBS was also established. From those actuals was extrapolated an exten- 
sive “knowledge base” for a wide range of vehicle stages and elements which were scaled from the 
Shuttle baseline. This knowledge base was developed utilizing engineering judgment and knowledge 
of the various new vehicle configurations and requirements. When two different type vehicles are in 
flow simultaneously, synergism of vehicle elements or stages is accounted for and common use of 
facilities, if appropriate, is likewise considered in the resulting cost and schedule estimates. 

This model and Users Guide have been delivered to KSC as of the contract end and the GOCM 
has been in actual use at KSC for some several months. The model has been beta tested at KSC and the 
results have been excellent. 

2.0 PRICE Calibration for NASA Users 

SAIC was requested to support efforts of the NASA Headquarters Comptroller Office and 
PRICE Systems in calibrating the commercially available PRICE Cost Model for NASA users. This 
effort had been underway with PRICE Systems prior to our involvement but little headway was being 
made. SAIC’s efforts were primarily to collect, normalize, and provide historic cost and technical data 
on NASA projects to PRICE; to interview PRICE users at all NASA centers and determine their needs 
and concerns relative to using PRICE as a cost estimating tool; and to write briefs and reports to be 
distributed to all NASA PRICE users on topics relative to calibration procedures. 

SAIC developed a survey form, interfaced with NASA centers by visit, telecon, or through the 
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survey form and determined who the PRICE users were at each NASA installation, what their unique 
needs were, how calibration was currently being accomplished, and what they considered to be prob- 
lems or shortcomings with the PRICE model. This effort resulted in two report deliverables which 
were distributed NASA-wide: “GE-PRICE Enhancements — Survey Results” and “GE-PRICE Enhance- 
ments — Selected Support Tools and Methodologies”. 

SAIC researched, wrote, and published NASA-wide two Technical Briefs: “The Data Collec- 
tion Process” and “PRICE Schedule Adjustments to Reflect a NASA Environment”. These briefs pro- 
vide NASA PRICE users information and guidance as to ways to best utilize the PRICE model. In 
addition, SAIC collected, normalized, documented, and provided PRICE Systems with subsystem level 
technical descriptions on AMPTE-CCE, CRRES, COBE, HEAO-1, HST-SSM, Landsat- 1 , GRO, Ex- 
ternal Tank, ERBS, Galelileo Orbiter & Probe, Magellan, and UARS. These data were to allow PRICE 
Systems to attempt to calibrate their data base to reflect NASA projects and tie costs to these technical 
parameters. SAIC also proposed a simplified, subsystem level approach using NASCOM data for cali- 
brating NASA projects. Data and methodology to exercise this approach is planned to be developed in 
a follow-on SAIC effort for NASA. 

3.0 Microgravity Experiments Cost Model (MECM) 

SAIC developed and delivered an entirely new cost model for the Microgravity Projects Office. 
This was a two year effort which ended June 30, 1994 and included cost, schedule and technical data 
collection, analysis and normalization; WBS development; data stratification, regression and CER 
development; computer model structuring and development; model analysis and beta testing; model 
and data base documentation and Users Guide preparation; and actual microgravity project cost esti- 
mating. 


The initial data collection efforts centered on MSFC microgravity projects, but was expanded 
to include JPL and LeRC microgravity projects as well. Data at varying levels of detail on a total of 52 
microgravity experiments was collected. Because of the need for low level data, an additional 75 space- 
craft data points were added to the data base. After normalization, the data was matrixed against 74 
different technical, schedule or cost parameters. A standardized WBS consisting of seven subsystems 
and 21 components was also established and all data was displayed against this WBS. Statistical re- 
gressions were run and CERs were established for DDT&E and flight unit costs utilizing various tech- 
nical parameters including weight, volume, power, temperature range, etc. The total applicable data 
base was used to establish each CER slope, but two curves, spacecraft and microgravity, were devel- 
oped so that microgravity’s lower costs could be accounted for. 

The automated cost model was developed in a user-friendly manner with Help buttons, warning 
messages, a straight forward estimating approach, and excellent documentation. Both IBM and Macintosh 
versions of the model were developed. 

A schedule estimating feature was added to the model which would accept user schedule inputs 
or generate its own schedule based on microgravity historical data. In addition, a cost penalty/benefit 
for schedule adjustments calculation was programmed into the model so that overly optimistic or pes- 
simistic schedules impact the resulting cost estimate. 

Finally, a cost risk feature was developed which provides risk profiles and contingency alloca- 
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tion for all appropriate cost elements in an estimate. The analysis considers the risk associated with the 
CERs themselves and the risk associated with the technical inputs to the CERs. 

The automated model, documented data base, and users manual have all been delivered to 
MSFC and the model has been in use for several months. Training classes on the use of the model were 
also conducted by SAIC for a number of MSFC microgravity project managers and business manage- 
ment personnel at the time of model delivery to MSFC. 
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